(N 



c^ 



S 



Optimal stopping of strong Markov processes 

Soren Christensen*, Paavo Salminen"^, and Bao Quoc Ta ^ 



O 

^ ■ Abstract 

Qh' We characterize the value function and the optimal stopping time 

■^ . for a large class of optimal stopping problems where the underlying 

CSJ I process to be stopped is a fairly general Markov process. The main 

result is inspired by recent findings for Levy processes obtained essen- 

q/ . tially via the Wiener-Hopf factorization. The main ingredient in our 

PLh ! approach is the representation of the /3-excessive functions as expected 

r~| ' suprema. A variety of examples is given. 
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t^ : 1 Introduction 

^ ■ Consider a real-valued strong Markov process X = {Xt)t>o- Let P^; and E, 

^ . stand for the probability and the expectation, respectively, associated with 

X when initiated from x. The natural filtration generated by X is denoted 

by J-" = {J^t)t>o and A4 is the set of all stopping times with respect to J-". 
^^ I We are interested in studying the following optimal stopping problem: 

Find a function V (value function) and a stopping time r* (optimal stopping 

time) such that 

V{x) := supE^(e-^^G{Xr)) = Eje-^"*G(X, 
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where the function G (reward function) is assumed to have some regularity 
properties to be specified later. On {r = 00} we define 

e-^^G{Xr) = limsupe-'^*G(Xt). 

t— i>oo 

Recall that a non-negative, measurable function u is called /3-excessive for X 
if the following two conditions hold: 

Eje-^'u{Xt)] < u{x) \/t>0,xeS, 



limEJe-^*M(Xi) j = u{x) \/ x e S. 

A basic result, see Theorem 1 p. 124 in Shiryayev [25j, is that if the reward 
function G is lower semicontinuous then the value function V exists and is 
characterized as the smallest /3-excessive majorant of G. 

During the last decade the theory of optimal stopping for Levy processes 
has been developed strongly. This research has been focused on analyzing 
the validity of the smooth pasting condition, see Alili and Kyprianou [T], 
Christensen and Irle [S], on the one hand, and solving particular problems, 
see Boyarchenko and Levendorskii [1], Mordecki [TTj, Novikov and Shiryayev 
|19j . |2U] . Kyprianou and Surya [T3], Surya [2B], Mordecki and Salminen [TS] . 
Deligiannidis, Le and Utev ^ , on the other hand. The methodology to derive 
explicit solutions is in many cases based on the Wiener- Hopf factorization. 

For random walks in discrete time optimal stopping problems with the 
reward functions G{x) = x"*" and G{x) = {e^ — 1)~^ were solved already by 
Darling et al. in [S]. In Mordecki [17] the results in [S] were lifted to the 
framework of Levy processes with the focus to the reward functions G{x) = 
{K — e^)~^ (put option) and G{x) = {e^ — K)~^ (call option). Furthermore, for 
the value function G{x) = {x^Y, z/ > 0, the problem was solved by Novikov 
and Shiryayev in [12] and [20] for random walks and Levy processes using 
Appell polynomials and Appell functions, see also Kyprianou and Surya [15] 
and Alili and Kyprianou [IJ. In addition to the Wiener-Hopf factorization, 
the fluctuation identities constitute an essential tool in these solutions. 

This technique was generalized under certain conditions to be applicable 
for more general reward functions independently in Surya PS] and Deligian- 
nidis et al. [5]. In particular, in [2B] it is assumed that there exists a function 
Q such that the reward function has the representation 

G{x)=Eo(q{x + Mt 

where T is an exponentially with parameter /3 > distributed random vari- 
able independent of X and Mt := supg<^<(Xs is the running maximum of 



the Levy process X. The function Q should have a unique positive root x* 
such that Q{x) < for x < x* and Q{x) > and non-decreasing for x > x*. 
Then the stopping problem Q is, in fact, one-sided and the optimal stopping 
time is 

T* := inf{t : Xt > x*}. 

Moreover, the value function is expressed in [26j as 

V{x) = Eo {Q{x + Mt)1{.+Mt>.*}) . 

In in] the starting point is the assumption that there exists a function g 
such that 

G{x)=Eo(g{x + XT)). (2) 



To see the connection between Q in ([T]) and g in 1^ recall the Wiener-Hopf 
factorization 

Xt ^^ Mt + /^, (3) 

where I^ = It '■= info<i<r-^t and independent of X. Consequently, 



Qix)=Eo[^gix + lT)j. (4) 

Since in [2] the stopping problem is analyzed via the function 

x->Eo(^(x + /t)) 

the relationship (JH) implies that the results in [2S] and [5] are very close to 
each other. 

Our main result characterizes the solution of the optimal stopping prob- 
lem Q similarly as in [T^], [25] and [H] but is valid for very general strong 
Markov processes, e.g., for diffusions. Levy processes and continuous time 
Markov chains. The main ingredient in our approach is the representation 
of /3-excessive functions in the form 



X H-Ea; sup f{Xt] 

\0<t<T 

for a function / > 0. This kind of representations have been studied in 
Follmer and Knispel ^TT] and El Karoui and Meziou [TU] . 

The structure of the paper is the following: In the next section we state 
and prove the main theorems which lead to explicit solutions to the optimal 
stopping problem Q for general Hunt processes X and reward functions G 
with some representation properties. Depending on the form of the repre- 
senting function / the optimal stopping time is one-sided or two-sided. We 
furthermore present in section 2.2 a way for finding / explicitly. In section 
2.3 the connection to the representing measure approach is discussed. In 
Section [3] the results are illustrated with a variety of examples. 



2 Main results 

2.1 Verification theorems 

We use the notation from the introduction and let X = {Xt)t>o be a Hunt 
process taking values in a subset S of M, e.g., an interval or a finite set. 
We refer to [IB] for references and further discussions on Hunt processes. 
For a full account see Blumenthal and Getoor ^J, Chung and Walsh [7| 
and Sharpe [23]. In particular, X has the strong Markov property and is 
quasi left continuous with right continuous sample paths with left limits. 
For simplicity, we suppose that the lifetime of X is a.s. infinite. 

Remark 2.1. In some problems it is of interest to study X, e.g., up to the 
passage time of a level. In such cases X is viewed as a Hunt process with 
lifetime equal to the passage time. For instance, a Brownian motion X killed 
at the first passage time 

Co := inf{t > : Xi < 0} 

is a Hunt process with lifetime Co- It is not difficult to see how to refine our 
approach to cover these and more general lifetimes, and we leave this task to 
the interested reader. 

Throughout the paper, the notation T is used for an exponentially dis- 
tributed random variable assumed to be independent of X and having the 
mean 1//3 where /3 > is the discounting rate introduced in Q. We remark 
that a function u is /3-excessive if it is 0-excessive, i.e, excessive, for X killed 
at T. 

The following lemma is the key to our approach for the optimal stopping 
problem Q. It is proved similarly as Proposition 2.1 in ^llj. We give a short 
proof for the sake of completeness. 

Lemma 2.2. Let f : S ^^ M.+ he an upper semicontinuous function and 
define u{x) := E^f supo<t<7- f{Xt)j. Then the function m : 5* i— )■ M U {+00} 
is (3-excessive. 

Proof. By the upper semi continuity of / and the right continuity of the 
sample paths of X the function t h- )■ supo<5<i f{Xs) is right continuous and. 



hence, measurable. It follows that u is well-defined. Consider for t > 

u{x) =E^( sup f{Xs 

\0<s<T 

> eJ sup /(X,);t < t) = eJe,( sup /(X,)| J-^); t < T 

\t<s<T / V t<s<T 



= E,(^E,( sup f{X,)\Xt);t<Tj =E,[u{Xty,t < T 

= E,(e-^'u{Xt 
and, therefore. 



E^[uiXt);t<Tj <u{x). 
Next, 

limEje-^'u{Xt)) =limE,f sup f{X,);t<T 

t->-0 \ / *-^0 \t<s<T 



= EJ sup /(X, 

\0<s<T 

= m(x), 
where the second equality holds by the monotone convergence. D 

Example 2.3. Consider a regular linear diffusion X on M. Let x — ?■ ^^(x, y) 
he the [^-resolvent kernel associated with X. As is well-known, the function 
X — )■ up^x.y) is [5 -excessive for any fixed y . We find the representation up{- , y) 
as expected supremum. Recall that 

E4e-^«.) = ^ = I If- I '^y- I (5) 

V ^ up{y,y) [ ^, zf x>y, J 

where Hy := inf {t : Xt = y} and ipp and ipp are the fundamental increasing 
and decreasing, respectively, solutions for the generalized differential operator 
associated with X , see 13]. Introducing 

x^ f (x) = [ "/3(^'^)' '^ ^ = yA 

■'y^ ' \ 0, otherwise, J 

we have hy ^ 



EJsup/,(Xi) =up{y,y)¥^{Hy<T) 

V t<T J 

= up{y,y)E^{e-^''y) 
= up{x,y). 



For another example, consider the P-excessive function ipji A 1. We assume 
that there exists a point y such that ip/siy) = 1- Notice that ipi3{x) > 1 for all 
X > y since ip/^ is non- decreasing. Define 

X -^ fli^) = l[s/,+oo)(a;). 
Then, calculating as above, we have 

{ I, if x>y,\ 



We consider now the optimal stopping problem Q and assume through- 
out the paper that the reward function G is non-negative, lower semicontin- 
uous and satisfies 

Ejsupe-^*G(Xi)) <oo (6) 

Our main result which characterizes solutions of Q is as follows: 

Theorem 2.4. (One-sided case). Assume that there exists an upper semi- 
continuous function f : S ^^M. and a point x* E S such that 

(a) (i) f{x) <0 for X < X*, 

(a) f{x) is positive and non- decreasing for x > x* . 



(h) (i) E, (^ supo<t<r /(Xt) j = G{x) for x > x* 
(a) E^Uupo<t<T/TOJ > G{x) forx < x* . 
Then the value function of the optimal stopping problem (^ is given by 

V{x) = eJ sup /(X,)l{x.>x'}) = E, (/(Mt)1{m^>.*}) (7) 

\0<t<T J ^ ' 

and 

T* ■= inf{t > : Xt > X*} 

is an optimal stopping time. 

Proof. Since the function x h-> f{x)li^x>x'} is non-decreasing, the second 
equahty in ([7]) holds. Let V denote the function given via these expressions, 
i.e., 

V{x):=eJ sup f{Xt)l{x,>.'}). 

\0<t<T / 



By Lemma 12.21 we know that V is /3-excessive, so that {e'^^V{Xt))t>o is 
a non-negative cadlag super martingale (see [7| p. 101). Hence by optional 
sampling we have for all x and all stopping times r 

V{x)>E,(^e-^^V{X,)y (8) 

By condition (jaj) we furthermore have f{x)l{x>x*} > f{x) for all x, therefore 
using (jb]) we obtain from ([S]) 



V{x)>E,[e-('^G{Xr] 
To conclude the proof we show that 

V{x)<Eje-''^*G{Xr^)). (9) 



The value function V being the smallest /3-excessive majorant of G it then 
follows that V = V and r* is an optimal stopping time, as claimed. To prove 
(ED consider 



V{x)=eJ sup /(XOl{x.>.'}) 

\0<t<T / 



EJ sup fiXt)l{x,>.^};r* <T 

\0<t<T 



E,[^f{MT)l{Mr>x'};r*<T 

eJ/( sup Xi);r*<r) 

\ T*<t<T / 



= EJe-^^*Ex,.(/(sup X,] 

\ ^ 0<t<T 

by the strong Markov property. For the last equality note that by the mem- 
oryless property of the exponential distribution 

eJ/( sup Xty,r* <t) 

\ T*<t<T / 

= ^"e,(^E,(^/( sup Xt)|J'..)l{..<.})p(Tecir) 
=eJex^, ( r f{ sup X,)P(T e dr)]e~P^' 

\ \J0 0<t<r J 

=Eje-^^*Ex,J/( sup Xi))). 



Since /(supo<i<r ^t) < supo<i<r f{Xt) we obtain 



V{x)<Eje-^^'Ex^J sup f{Xt] 
\ ^o<t<r 

/3r* 



= E,(^e-^^G'(X.: 

where we used ©(ii) in the last step. This shows (jH]) and the claim is 
proved. D 



Remark 2.5. By applying Theorem \2.4\ to the process —X, the reward func- 
tion X I— > G{—x) and to the function x h-> f{—x) one obtains the analogous 
result for a threshold to the left applicable, e.g., to put-type options. 

Although one-sided stopping rules as in the previous theorem often arise 
naturally in many examples (especially for monotonic reward functions and 
discounting, see also [6] for a general discussion), for other problems a two- 
sided stopping rule is needed. The following theorem gives the corresponding 
result for this case. 

Theorem 2.6. (Two-sided case) Assume that there exists a lower semicon- 
tinuous function / : S* i—> R and points x^, x* & S such that 

(a) (i) f{x) < for x^ < x < x* , 

(a) f{x) is non-increasing for X < x* and non- decreasing for x > x* , 

(b) (i) E^. (^ supo<t<r /(^t) j = G{x) for x ^ [x*, x*] , 
(a) E^f supo<t<T/(^t)j > G{x) forx G [x*,x*]. 

Then the value function V of the optimal stopping problem ([^j satisfies 
V{x)=eJ sup /(Xt)l{x,^[x.,x*]}) 

\0<i<T / 

=E,.(^/(/t)1{/,<x.} V /(Mt)1{m,>.*}) (10) 

and 

T* :=inf{t >0 -.Xt ^ [x„x*]} 

is an optimal stopping time. 



Proof. This theorem is proved similarly as Theorem 12.41 The second equality 
in fITU]) follows from the monotonic properties of /. Letting 



V{x):=E^[ sup fiXt)l{Xt^[^,,x*]} 

\ 0<t<T 

it is seen from Lemma 12.21 that V is /3-excessive and, by conditions (a) and 
(b), 

V{x)>EJe-^^GiXr) 



for all stopping times t E Ai. It remains to show that 

V^(x)<E,(^e-^^*G(X,0). (11^ 

For this, consider 



V{x)=Ej sup /(XOl{x.>.*} V sup /(Xi)l{x,<.4;r*<T 

\0<t<T 0<t<T 

= E,(^fiMT)l{Mr>x*} V /(/t)1{7^<...}; t* < r) 
= E. (/( sup X,) V /( mi X,); r* < t) 

\ T*<t<T T*<t<T / 

= eJs-^^'Ex^^ (/( sup Xt)) V (/( inf X, 



where the last equahty is obtained by the strong Markov property. Since 

/( sup X,) V/( inf X,) < sup /(X,), 
o<t<r o<i<T o<i<r 

we have 

V{x)<EJe~^^''Ex^J sup f{Xt 

\ ^0<t<T 

= E,(e-^^*G(X.*))- 

Thus the proof is complete. D 

Remark 2.7. The methodologies in /P/ and Wd^ seem to he applicable only 
for one-sided problems (for Levy processes) . It is remarkable that the present 
approach provides a clean characterization also in two-sided cases (for general 
Hunt processes) . 



2.2 Computing / 

To apply Theorem 12.41 and 12.61 for a given reward function G the question 
arises how to find a function / fulfilhng the conditions stated therein. For the 
related problem for underlying Levy processes, Surya [2S] gave a general way 
for sufficiently regular bounded functions G using the Fourier transform, 
but this seems to be specific for Levy processes and is based on a certain 
assumption on the zeros of the Wiener- Hopf factors of X. We now discuss a 
more general way. 

Let G be a given reward function and assume that for all x 

^x U'^'G{Xt)\ ^0 as t ^ oo. (12) 

Note that, by dominated convergence, (IT^ is fulfilled if in addition to (P) it 
also holds 

lim Q-^^GiXt) = P:,.-a.s. 

i— ^oo 

Furthermore, assume that the function G is in the domain of the extended 
infinitesimal generator of X killed at T (cf. [21, VII. 1]). Then, by definition, 
there exists a function / such that 

/ |/(X,)|rfs<oo P,-a.s 
Jo 

and the process 

(e-^'GiXt) - G{Xo) + fe-^'f{X,)ds) 
\ Jo J t>o 

is a (J-t,Px.)-martingale. Consequently, 

E, (^e-^*G(X,)) - G{x) + E, (^|J e-^^~f{X,)ds^ = 0, 
and as t — )■ oo we obtain using flT^ 

G{x) = \iui¥.J f e-P'f{X,)ds) . 

t->oo \Jq J 

Hence, if, e.g., 

^-{j^ ^~^V{Xs)\ds^ <oo, 

the Lebesgue dominated convergence theorem yields 

G(x) = i E.. (/(Xt)) . (13) 

10 



Notice that for G in the domain of the infinitesimal generator A of X we have 
f = —{A — P)G. In this case equation flT51) reflects one of the fundamental 
properties of the resolvent operator. 

Remark 2.8. For optimal stopping problems for Levy processes it was as- 
sumed in 19] that a representation of type flT^ (see also ^) holds for G. 
Hence, the above considerations are useful when applying the methods in |^. 

Imitating the approach for optimal stopping of Levy processes, we next 
express G as the expectation of a function of Mt (cf. ([T]) and ([2])). Obviously, 
it holds 

E, (/(Xr)) = E,. (e, (/(Xt)|Mt)) . 

We assume that the regular conditional probability distribution 

P,(Xr e dy\MT = z) ■= P,(Xr G rfy , My G dz)/V,{MT G dz) (14) 

exists and has the needed regularity properties so that it holds 

-^E, (/(Xr)) = E, {q{Mt; x)) , (15) 

where 

Q{z- x) ■= )- r f{y)K{XT e dy\MT = z). (16) 

P J —oo 

These observations are now applied to characterize solutions for optimal 
stopping problems for Levy processes and linear diffusions via Theorem I2.4[ 
Levy processes are discussed first. The following lemma is well known. We 
present a short proof for the sake of completeness. 

Lemma 2.9. For a Levy process X (which is not a subordinator or a com- 
pound Poisson process) it holds that 

F,{Xt e dy\MT = z)= P,(/t G dy). (17) 

In particular, the conditional distribution does not depend on x. 

Proof. The claim follows from the Wiener-Hopf factorization ^ and the 
spatial homogeneity of X. Indeed, 

P^(Xt edy, Mt e dz) = Po(a; + Xt e dy , x + Mt e dz) 

= Po(a; + MT + lT^dy, x + Mt edz) 
= Po(2 + lTedy, x + Mt edz) 
= F,{lTedy)F,{MTedz). 

n 
11 



Proposition 2.10. Consider the optimal stopping problem (^ in case of a 
Levy process X (as introduced in Lemma \2.y\) . Assume that there exists a 
function f such that / f73]) holds and define 

Qiz):=^ f f(z + y)FoilTedy). (18) 

If Q fulfils (a) in Theorem \2.4\ then the value function of the optimal stopping 
problem (Qj is given as in ^ with f = Q \/ 0. 

Proof. Since Q is assumed to fulfil (a) in Theorem 12.41 also / fulfils this 
condition. We need to verify that / satisfies (b) in Theorem 12. 4[ Definition 
fllSp of Q, Lemma [2.91 and formula flT^ yield 

G{x) = ^E, (fiXr)) = E,(^Q{Mt] 
Consequently, since Q < f we have for all x 



G{x) = E^[Q{Mt)J < E,|/(Mt)J < E,(^ sup f{Xt)j. (19) 

In case x > x* - since Q satisfies (a) in Theorem 12.41 - (TT^ holds with 
equahties instead of inequahties, and this completes the proof. D 

Next we consider the optimal stopping problem (jfj for a regular linear 
diffusion. For simplicity, it is assumed that X is conservative and takes values 
on the whole M. The following result is extracted from P, 11.19]. We use the 
notation from Example 12. 3i 

Lemma 2.11. For a conservative regular linear diffusion X on M it holds 
that 

VAXr e dylMr = .) = .:"'^\'f^l , . (20) 

where m is the speed measure. In particular, the conditional distribution does 
not depend on x. 

The next result can now be proved analogously as Proposition I2.1U[ 

Proposition 2.12. Consider the optimal stopping problem (^ for a diffusion 
X as introduced above. Assume that there exists a function f such that l[TS\) 
holds and define 

Q^^y_ 1 Iloofiy)My)Hdy) ^21) 



If Q fulfils (a) in Theorem \2.4\ then the value function of the optimal stopping 
problem (^ is given as in ^ with / = Q V 0. 
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To apply Proposition 12.121 it is useful to have condition on hand to guar- 
antee that Q fulfills (a) in Theorem I2.4[ The following Lemma gives such 
a condition in terms of the function / which is known explicitly in many 
situations of interest. 

Lemma 2.13. In the setting of Proposition IKJB assume that there exists x 
such that f{x) < for x < x and f{x) is non- decreasing for x > x. Then 
Q < or there exists x* such that Q fulfils (a) in Theorem \2.4\ 

Proof. For simplicity we assume that the measure m is absolutely continuous, 
i.e. m{dy) = m{y)dy, and let 

X* = mi{z : / f{y)ipi3{y) m{y)dy > 0}. 

J —CO 

If X* = oo, then the claim trivially holds. Therefore, we assume x* < oo. It 
remains to prove that Q is non-decreasing for x > x*. Obviously x* > x and 
therefore by the assumption on / we get 

UXz)-f{y))My)m{y)dy>0. 

Using this fact, by differentiating the function Q we immediately see that 

£Qi^) > 0. □ 

Remark 2.14. It is critical that the function Q defined in l[T8\} for Levy 
processes and in [^) for diffusions does not depend on x. This clearly fol- 
lows from the fact that the conditional distribution of Xt given Mt does not 
depend on the initial state of the process. Path decomposition theorems for 
strong Markov processes imply such results in general, see Millar JTU]. This 
observation yields a method for finding f for general underlying processes. 

2.3 Connection to the representing measure approach 

Another approach to the solution of optimal stopping problems is the obser- 
vation that finding the solution of such problems is equivalent to finding the 
Choquet-type integral representation of the value function in terms of the 
resolvent kernel up{x,y). The Radon measure a that appears in this rep- 
resentation characterizes the /3-excessive value function V , and furthermore, 
the stopping region can be described as the support of a. 

This approach was first discussed in [22] for optimal stopping problems of 
diffusions. In [IS] a verification theorem based on this approach applicable 
for general Hunt processes is presented. Moreover, in [18j the appearance of 
a function of the maximum in the solution of optimal stopping problems for 

13 



Levy processes is explained via this approach and the Wiener-Hopf factor- 
ization. 

The critical point to utilize the approach is to find a candidate solution 
for the measure a. In [23], this was carried out explicitly for spectrally 
positive Levy processes in the Novikov-Shiryaev problem in terms of the 
Appell polynomials of Xt using the Laplace transforms. In Proposition 12.151 
we extend these results by pointing out a connection between this approach 
and our approach in this paper for a general Hunt process X which does not 
have negative jumps, that is, 

P^ ( inf (Xt - Xt_) > o] = 1 'ixeS. (22) 

We work in the setting of Hunt processes satisfying the set of assumptions 
as given in [THl Section 2]. In particular, it is assumed that there exists a 
dual resolvent with respect to a duality measure m and that Hypothesis (B) 
from [13] holds. Moreover, we assume that the Laplace transform of the first 
hitting time H^ '■= inf{t : Xt = x} admits for all Xo < x the representation. 



Eje-f^^^o) = M^ Vxo < X G S, (23) 



where T^ = inf{t > : Xt = x} . This representation is valid for a large class 
of processes. A sufficient condition is that (I) and (II) below hold true, see 
[21 Chapter V.3 and Exercise VI.4.18]. 

(I) The mapping x H- Jg Ujs^x, y) f {y)m{dy) is continuous, where / is a 
bounded and measurable function with compact support. 

(II) Every point x G S* is regular for {x}, that is 

P,(T, = 0) > 0, (24) 

where T^ = inf{t > : X^ = x}. 

Assumption (I) is (4.1) in [2, p. 284], and assumption (II) implies that X has 
a local time at every point of the state space. Condition (I23|) furthermore 
holds true whenever X is a spectrally positive Levy process, see ^Mj, Theorem 
3.12, Corollary 8.9]. 

Note that, in case X have positive jumps, overshoot occurs in the problem. 
Nonetheless it is possible to find an explicit representation of the measure a. 

Proposition 2.15. Let X be a Hunt process as described above with no neg- 
ative jumps that fulfills condition 1^) . Assume that for the optimal stopping 
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problem (^ there exists a function f such that l[T3\) holds. Moreover, assume 
that the function 

Qiz) := 1 /' /(y)P(Xr G dy\MT = z) 

fulfills (a) in Theorem \2.4\ Then the value function V of (^ has the repre- 
sentation 

V{x) = up{x,y)a{dy) V x G 5, 

j[x*,oo)ns 

where the measure a is given by 

a{dy) = I /(^)^(^^)' /«^ ^ > ^;' I (25) 

^ ^ 1 0, for y < X*, I ^ ^ 



and 



E,*(/(Xt)1{x,<.*} 



^(Kl) = V^ ^ -■ (26) 

^^ ^' PUfj{x*,X*) ^ ' 

In the particular case that X is a spectrally positive Levy process, then a({a;*}) 
0, and 

V{x) = E^[f{XT)l{Xr>.'}). 

Proof. We remark first that similarly as in the proof of Proposition 12.101 it 
can be verified that the value function V of the problem satisfies the equality 



V{x)=¥.^[Q{MT)l{Mr>.'}). 

where x* is the point associated with Q via (a) in Theorem 12 .41 Now we have 

V{x) =¥.,[q{Mt)1{Mt>.*]) = ^E,(^E,(^/(Xt)|Mt)1{a/^>..}) 
= ^E,(^E,(^/(Xt)1{m^>.*}|Mt)) = ^E,(^/(Xr)l{M^>.*}) 

= -n^x{fiXT)l{XT>x*}j + -j^x{fiXT)l{MT>x*,XT<x''}j- 

Since it is assumed that X moves continuously down we rewrite the second 
summand 

^x{f{XT)l{MT>x*,XT<x'-}] = ^x{fiXT)l{T^:,<:T,XT<x'-}j, 
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and, by the strong Markov property, 

Ex [fiXT)l{MT>x*,XT<x*}) = ^x [^x [f{XT)l{XT<x*}\J^T^,) l{r^*<T} 

= E, (e-^^^*Ex,^, [f{XT)l{Xr<x'})) 
= Ex* {f{XT)l{Xr<x*}) E, (e-'^^^*) . 

Letting c = E^,. f /(Xt)1{Xt<x*}) and using assumption (125]) we obtain 



V{X) = ^E,(^f{XT)l{Xr>x*}) + ^^x(^fiXT)l{Mr>x*,Xr<x'}) 
~ C 

f{y)ui3{x,y)m{dy) + - — — — -U[s{x,x*) 
{x*,oo)r\S pui3{x*,x*) 

up{x,y)a{dy) 

a;*,oo)n5 

with 0" as in fl2Sl) and in fl2Bl) . It remains to prove that a is indeed non- 
negative. To this end, we use the fact that each /3-excessive function u 
locally integrable with respect to the duality measure m can be decomposed 
uniquely in the form 

u{x) = / up{x,y)au{dy) + K{x), (27) 

where au is a non-negative Radon measure and hu is a /3-harmonic function, 
see [T^ Theorem 2 p. 505 and Proposition 13.1 p. 523]. Since the value 
function V is /3-excessive we may apply ( 1271) to obtain 



Ufs{x,y)av{dy) + hv{x) = V{x) = / up{x,y)a{dy), 

J[x*,oo)r\S 

where o"y is the non-negative Radon measure associated with V . By the 
Hahn decomposition we can write a — ay = a^ — a~ for some non-negative 
measures a^ and a~ . Therefore, 

hv{x)+ up{x,y)a~{dy)= j up{x,y)a^{dy). 



By the uniqueness of the decomposition in (I27|) we obtain /ly = and a"*" = 
a~ . Hence, a = ay is indeed a non-negative measure. This proves the first 
claim. 

For spectrally positive Levy processes it is well known that —It is Exp{X)- 
distributed, where A is the unique positive root of the equation 
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Therefore, by the definition of Q and Proposition [T7] we see that Q is con- 
tinuous. In particular, Q{x*) = 0. Using the Wiener-Hopf factorization ([3]) 
we obtain 



= J^^ Eo(/(x* + t + /°)l|i+,o<o|)p(MT e dt). 

Therefore, E^. (/(Xt)1{Xt<x*}) = 0, since 

Eo(/(x* + t + /^)l|i+,o<o^) = r f{x* + t + s)\e'''ds 

= e-^*r f{x* + y)\e^My 

J —oo 

= e-^'Qix*) = 0. 
Consequently, 



Ex* /(^r)l{Xr<i*} 



D 



Remark 2.16. By inspecting the previous proof one can see that assumption 
can be weakened by assuming that ([22D holds for xq = x* only. 



3 Examples 

In this section we demonstrate the applicability of the above theory by solving 
some optimal stopping problems for diffusions. Levy processes, and continu- 
ous time Markov chains. 

3.1 Ornstein-Uhlenbeck process 

Let X be an Ornstein-Uhlenbeck diffusion with parameter 7 > 0, i.e. the 
differential operator associated to X is 

Af = -—f-jx-f. 
2 dx'^ dx 

We consider the optimal stopping problem Q for X with the reward function 

Gn{x) = {x^)\ neN. 
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For general Levy-driven Ornstein-Uhlenbeck processes it was proved in [B] 
that the optimal stopping rule is one-sided, but no explicit way for finding the 
boundary explicitly in general was given there. Here we want characterize the 
optimal boundary for the Ornstein-Uhlenbeck diffusion using our approach. 
We have 



Using Ito's lemma it is seen that /„ indeed has the mean value property for 
Xt, i.e., 



Recall that for Ornstein-Uhlenbeck processes the speed measure m is given 

by 

ra{dx) = 2e '''^ dx 

and 

^;3(x)=e^^'/2p_^/,(-xv/2^), 

where Vi, denotes the parabolic cylinder function, see [21 Al.24,II.7]. Defining 
Qn = Q as in fl?Il) we obtain 



x" = E,(^Q„(Mt)). 



Using arguments similar to those in Lemma 12.131 it is easy to see that Q 
has a unique positive root x*, which is given by the positive solution of the 
equation 

^' f{y)e-^'"^'V_^/,{-yV2^)dy = 0. (28) 



An approximative value of x* can be found using numerical methods. Notice 
that Qn can also attain positive values on the negative half-line. However, 
this problem can be dealt with by considering the function 

fix) = l °' ^-°'l 
•^^"^^ \ Q{x), x>0, / 

which obviously satisfies the requirements of Theorem 12.41 Therefore, we 
obtain that the stopping time 

T* = inf{t : Xi > X*} 

is optimal. 
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3.2 American put in a Levy market 

In this subsection we review the well known example of a perpetual American 
put option in a Levy driven market in the light of our approach (cf. the 
solutions in [TTj and pQ, see also [5] for further discussions). Although the 
solution is well-known we want to demonstrate how to find the representing 
function / systematically with the approach discussed in Subsection I2.2[ 
Since the reward function G{x) = K — e^,K > 0, is non-increasing we use 
the modified version of Theorem 12.41 see Remark 12.51 The American call 
option can be handled similarly. 
The generator A of X is given by 

where {b, c, vr) is the Levy triple of X and / is assumed to be in the domain 
of A. We wish to apply P — A to G{x) = K — e^'. For this ansatz to work we 
assume that 

Eo(e^i) < e^. (30) 

At the end we will see that the formula obtained is a solution for general X. 
We have 

(/5 - A)G{x) = I3{K - en +e^(^ + b + jT (e^ - 1 - yl^^^^Mdy)] 

= /3ir-(/3-V^(l))e-=:/», 
where ipi^l) = logEo(e"^i) < f3. To find / we calculate, cf. Proposition 12. lUl 



(31) 



Q{x) = - Eo [f{x + Mt)) =K- " ^' ^ Eo(e^^^^)e 

By noting that 

/3-V^(l) 1 



/3 Eo(e^^) 

and applying the Wiener-Hopf factorization ([2]) we obtain 

Q(x) = ir-Eo(e^^)-^e^ 

Now we have a candidate for / by our ansatz, i.e., / = QVO, and formula 
flHT]) also makes sense for general Levy processes without the integrability 
assumption flSP]) . Indeed, one immediately sees that for general X we have 



K-e^ = ¥.^[Q{lT] 
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We can apply Theorem 12.41 (together with Remark 12.51) to / and obtain that 
X* = log(A'Eo(e^^)) is the optimal stopping boundary and the value function 
is given by 



3.3 Novikov-Shiryaev optimal stopping problem 

In [12] Novikov and Shiryaev solved the optimal stopping problem Q with 
Gn{x) = (x^)", n = 1, 2, ... , for random walks and in [15] Kyprianou and 
Surya found the solution for Levy processes. Here we discuss shortly how 
the solution can be found with the methodology presented in this paper. For 
simplicity, we assume that /3 > 0. 

Let X be a Levy process with the generator A as given in 0291) . We fix 
the exponent n G {1, 2, . . . } for the reward function Gn and assume that the 
Levy measure vr satisfies 



/. 



\y\Mdy) < oo. (32) 

(-cx),-l)U(l,+oo) 

This condition implies that E^dXjl'^) < oo for all x, t > 0, and k = 1,2, . . . ,n. 
Using the independence of increments, it is easily seen that Ea;(X") is a poly- 
nomial in t. Consequently, also the moments of Xt up to n exist. 

Remark 3.1. In fT5^ the solution is found under the weaker condition that 
(only) the integral over (1, +oo) in (E^ is finite. Under this weaker condi- 
tion the moments of Mt up to n exist but the moments of It rnay not exist. 
Working under the stronger condition reveals the role of the Appell polynomi- 
als of Xt which we find interesting and important. In fact, since the function 
Qni see [JB) . that is, the Appell polynomial associated with Mt, found in this 
way satisfies conditions (a) and (b) in Theorem \2.4\ it is clear that our so- 



lution is indeed valid under the weaker condition and, hence, coincides with 
the solution in by Novikov and Shiryaev. 

We follow the receipt in Section U^ and operate with [i — A on x". This 
yields 

Ux) := (/3 - A) (x") 

^,x-(^.(.-l)x- + 5.x-) 



- j Ux + yY - x" - l/nx"-il{|,|<i}) 7r(rfy), 
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where the integral term is well defined due to 
order polynomial and satisfies the DE 



d_ 
dx 



fn{x) =nfn-l{x) 



Clearly, /„ is an nth 
(33) 



with /o = 1. We claim that 



Ej/„(Xr) =^Jfn{XT + X 



/3x^ 



(34) 



which, then, together with ([32D shows that x H- fn{x)/f3 is the nth order 
Appell polynomial associated with Xt, (for properties of the Appell poly- 
nomials, see [ISj and [23] )• To prove fl55]) we use standard computations, 
legitimate by ([S2D, with the Ito formula for Levy processes, see, e.g., [121 p. 
612], to obtain 



-^*E, {XI 



n\ n 



E, 



-/3s 



-n{n - 1)X;-^ + bnX^ 



^n—l 



px: 



ds 



+ ^x\ I e 



(X, + yr - X: -ynXr%y\^,} n{dy) ds . 



Multiplying both sides with /3, letting t — > oo and using that the moments 
of Xt are polynomials in t we obtain 



-/3x" = E, (j^' 



/3e" 



-n{n - 1)X;-^ + bnXl 



-n-l 



px: 



ds 



E, 



(3e 



-ps 



(X, + yf - X: -ynX:-%y^^,y\ 7r(dy) ds 



which proves (1341) . Next we apply Proposition I2.1UI to find the function Qr, 
and then /. We write formula (1181) therein as 



Qn{x)=f Q^^-r\x + y)¥,{lTedy), 
where Qi^'^^x) := fn{x)/f3. Recall from [23| the identity 



(35) 



Q 



{Xt), 



n 



+ y) = Y.[:]Qr\-)Qt'~i(^y) 



iUt), 



(36) 



A;=0 



where Q\ ^ and Q\ ^ denote the zth Appell polynomials of Mt and It, 
respectively. Identity flHBl) follows easily from the Wiener-Hopf factorization 
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and the definition of tlie Appell polynomials. Substituting (15^ into (15^ and 
using the normalizations 

gf,^^^ = 1 and Eo( Q\^^\It)] = for all i = 1,2, ...,n 

yields Q„ = Q^^'^^. Hence, our approach leads to the same characterization of 
the optimal stopping time in terms of Q^^^^ as in [TH]. The crucial property 
to be able to apply Theorem 12 ■4[ i.e., that Q^'^^ has a unique positive zero x* 
and is non- decreasing for x > x*, is proved in ^19j Lemma 5. Consequently, 
it now easily follows that 

^^^■~ 1 0, x<0, j 

satisfies conditions (a) and (b) in Theorem 12.41 and the solution of the optimal 
stopping problem results. 

Note that Proposition 12. 151 together with the fact that that x i— )■ fn{x)/P 
is the nth order Appell polynomial associated with Xt reproduces the for- 
mula of the representing measure for spectrally positive Levy processes ob- 
tained in [231 Corollary 8]. 



3.4 Absorbing Brownian motion 

Let X be a Brownian motion on ]R_|_ absorbed at 0, i.e., Xj = for t > Hq. 
We solve the optimal stopping problem Q with Gn{x) = x"', n > I. The 
first step is to find /„ such that (TT^ holds. Guided by Subsection 12.21 we set 

fnix) = (/3 - l^,)Gn{x) = X--' (^X' - !^t_^\ =; ^p^^^), (37) 

Instead of checking (1131) via a direct calculation, we recall that P„ is the 
nth order Appell polynomial associated with a standard Brownian motion 
(Wt)t>o started from and evaluated at T, see [231 Example 3.9]. Hence, for 
all X 

x^ = Eo(p„(x + Wt)) = E.,(^P4Wt)^ = V.(^/„(W^t) 

and, by the strong Markov property. 



E,[UWt)) =E,[^fniWT)l{T<Ho}) +^.[fniWT)liT>Ho} 
= ^x{ fn{XT)l{XT>0} 
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Consequently, we have 

G„(x) = ^E,(/„(Xt)1{x,>o} 

which is (IT51) a n y^ 2. To cover also the case n = 2 we abandon (IT5|) and 
instead try to find a function Qn such that 

E..(/„(Xt)1{x^>o}) =E,.(^Q„(Mt 

For this, the conditional distribution of Xt given Mt is needed, see (TTB]) . 
Since X is not conservative, this distribution has an atom at and, hence, 
the formula in Lemma 12.111 is not valid. However, from P, 11.19] we may 
deduce 

and 

P,(Mt G dz,XT G dy) = 2^i^^^^^$^dzdy. 



where il^pi.^) = sh.{xy/2P), x > 0. see ibid. p. 121. Consequently, the 
conditional density of Xt given Mt is given by 

UMMt = z) = ^^. 
and, therefore, 



P Jo ^i^{z) 

= ^ ^^^ ^^ ry-~^ (py^ - <''-^) \ ^^yM)dy. 

f3y/2^ch{zy/2^)Jo ^ y"^ 2 ; ^^v f^' ^ 



Since the integrand changes its sign from — to + one can see (after some 
easy calculations) that Qn satisfies ([a]) of Theorem 12. 4[ Although Proposition 
12.121 is not directly applicable it can be proved analogously as therein that 
(jb]) of Theorem 12.41 holds for /„ := Qn V 0. Therefore, the optimal stopping 
boundary x* is given as the unique positive solution to the equation 

[ y""-' {(iy' - !^(!^^ sh(y v^)rfy = o. (38) 

Integrating by parts, (155]) is seen to be equivalent with 

z ch(z V^) - -^ sh(2; V^) = 0. (39) 
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Notice that for n = 1 this equation does not have a positive solution and 
the optimal stopping rule is to stop immediately - in other words, reward 
G{x) = X is /3-excessive. 

It is also interesting to observe that fl39p is of the form 

dtpjz) .dGJyZ) 
GJz) — ; ^p{z) : = 0, 



dz 



dz 



and, hence, our approach yields in this case the same characterization of the 
stopping point as given in Remark (ii) p. 97 in |22]- 

3.5 Markov chain 

To see the general applicability of the theory we treat in this subsection an 
easy example with an underlying finite-state Markov chain. Let {Xt)t>o be a 
continuous-time Markov chain with state space E = {1, ...,4} and transition 
rates as given in the following figure. 




To obtain explicit results let us assume that /3 = 1 and G{1) = 3, G{2) = 
1 = ^(3) and G'(4) = 2. We could use the method described in Subsection 
2.21 to find / such that G{x) = Kx{sup^^2' fi-^t)) for all x & E, but it may 
be instructive to find it directly: Since 4 is an absorbing state, we take 
/(4) = G(4) = 2. To find /(3) we make the ansatz that /(3) < /(4) and 
obtain 



1 = G(3) = E3 (sup f{Xt)\ = /(4)P3(53 < T) + f{3)Fs{Ss > T), 

where the sojourn time S3 at state 3 is exponentially distributed with pa- 
rameter 2 and independent of T. Solving for /(3) yields 



/(3) = ^i±^a-2- ^' 



By a similar calculation we obtain that 



-1. 



/(2) 



{(5 + \2f (. .,.. A 



/32 + 2/3 A2 



1 - /(4) 



(/3 + A2) 
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Since 1 is a global maximum point of G taking 

/(I) = G(l) = 3 

yields the right representation. We see that the assumptions of Theorem 12.61 
are fulfilled (with x* = 2, x* = 4). Therefore 

r* = inf{t > : Xi ^ 3} 

is an optimal stopping time. 
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